Unsupervised learning by examples: On-line versus off-line.

We study both on-line and oo-line learning in the following unsupervised learning scheme: p patterns are sampled independently from a distribution on the N-sphere with a single symmetry breaking orientation. Exact results are obtained in the limit p ! 1 and N ! 1 with nite ratio p=N. One nds that for smooth pattern distributions, the asymptotic behavior of the optimal oo-line and on-line learning are identical, and saturate the Cramer-Rao inequality from statistics. For discontinuous pattern distributions on the other hand, the optimal on-line algorithm needs (at least) twice as many examples asymptotically to reach the optimal oo-line performance.